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Abstract 


Successful detection of weak signals is a universal challenge for numerous technical and 
biological systems and crucially limits signal transduction and transmission. Stochastic res¬ 
onance (SR) [l] has been identified to have the potential to tackle this problem, namely to 
enable non-linear systems to detect small, otherwise sub-threshold signals by means of added 
non-zero noise. This has been demonstrated within a wide range of systems in physical, 
technological and biological contexts [illle]. Based on its ubiquitous importance, numerous 
theoretical and technical approaches aim at an optimization of signal transduction based on 
SR |12L I13| . Several quantities like mutual information, signal-to-noise-ratio, or the cross¬ 
correlation between input stimulus and resulting detector response have been used to deter¬ 
mine optimal noise intensities for SR. The fundamental shortcoming with all these measures 
is that knowledge of the signal to be detected is required to compute them. This dilemma 
prevents the use of adaptive SR procedures in any application where the signal to be detected 
is unknown. We here show that the autocorrelation function (AC) of the detector response 
fundamentally overcomes this drawback. For a simplified model system, the equivalence of 
the output AC with the measures mentioned above is proven analytically. In addition, we 
test our approach numerically for a variety of systems comprising different input signals and 
different types of detectors. The results indicate a strong similarity between mutual informa¬ 
tion and output AC in terms of the optimal noise intensity for SR. Hence, using the output 
AC to adaptively vary the amount of added noise in order to maximize information trans¬ 
mission via SR might be a fundamental processing principle in nature, in particular within 
neural systems which could be implemented in future technical applications. 


Stochastic resonance, a phenomenon first described 
by Benzi et al. in 1981 [T], has been identified to 
have the potential to enable non-linear systems to 
detect arbitrary weak, otherwise sub-threshold sig¬ 
nals by means of added noise laiaiz]. Already in 
1995, Wiesenfeld and Moss have noted in their sem¬ 
inal paper that SR may be demonstrated within a 
wide range of non-linear systems in physical, tech¬ 


nological and biological contexts. In addition, they 
have demonstrated the existence of an optimal, non¬ 
zero intensity for the added noise, allowing maximiza¬ 
tion of information transmission ffU- To date, the 
phenomenon receives even increasing attention, es¬ 
pecially within the context of experimental and com¬ 
putational neuroscience [iums]. 

In smart non-linear signal detection systems based on 
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SR, the optimum noise level would have to be con¬ 
tinuously adjusted via a feed-back loop, so that the 
system response in terms of information throughput 
remains optimal, even if the properties of the input 
signal change (figure 1). For this processsing princi¬ 
ple the term adaptive SR has been coined miiiiis]. 
An objective function frequently used in theoretical 
approaches is the mutual information (MI) or the MI 
rate, respectively, between the sensor response and 
the input signal ismiiisi- The choice of the MI as 
objective function is natural since the fundamental 
purpose of any transducer is to transmit information 
into a subsequent information processing system. It 
has been shown previously that the MI as a func¬ 
tion of noise intensity has a well-defined peak that 
indicates the ideal level of noise to be added to the 
input signal |I4j . However, a fundamental drawback 
of the MI is the impossibility of calculating it in any 
application of adaptive SR where the signal to be de¬ 
tected is unknown. The same is true for a number 
of alternative objective functions, such as the signal- 
to-noise ratio (SNR) [T71 HU [HI El [3 [S] or the cross¬ 
correlation (CC) m [3] of the input stimulus and the 
resulting system response. Although finding the opti¬ 
mal level of noise becomes less important with arrays 
of transducers |3], it still remains an unsolved prob¬ 
lem for single detector systems. 

We argue that this fundamental drawback can be 
overcome by another objective function, namely the 
AC of the detector response, which leads to similar 
or even identical estimates of optimal noise intensi¬ 
ties for SR as the aforementioned objective functions, 
yet with the decisive advantage that no knowledge of 
the input signal is required (figure I). 

We introduce the concept of the success probability 
and prove analytically for bipolar signals and a mem¬ 
oryless detector with symmetric thresholds that the 
output autocorrelation Cyy{T=l) yields identical op¬ 
timum noise levels as the mutual information I{S; Y) 
and the cross-correlation of signal and output Csy 
(see methods section). Remarkably, all these funcions 
can be expressed as strictly monotonous functions of 
the success probability (for a detailed derivation, see 


methos section): 



Csy 

= 2Q-1 

(1) 

I{S-Y) 

= 1 + Q log2 Q + Q log2 Q 

(2) 

Cyy{T = i) 

= CssiT = l){l-AQQ) , 

(3) 


where C'ss(r = 1) is the autocorrelation for lag time 
r = 1 of the input signal, Q is the success probability, 
and Q = 1 — Q. 

Although these analytical results have been derived 
in the limited context of a simplified model, we show 
numerically that the concept holds true for a surpris¬ 
ingly wide range of SR systems. We validated our 
approach - exemplarily for the comparison of AC and 
MI - for a variety of models comprising different dis¬ 
crete and continuous input signals, different types of 
detectors and different signal-to-threshold distances 
(figures 2 and 3). As aperiodic input signals, three 
synthetically generated time series were used (figure 
2AI-3). A wave file of recorded speech was taken 
as an example for natural input signals (figure 2A4). 
A sine signal served as periodic input (figure 2A5). 
We modeled four types of memoryless detectors with 
discrete or continuous response functions and sym¬ 
metric or asymmetric thresholds (figure 3 insets), as 
well as a leaky integrate-and-fire neuron as a detector 
with memory. The model shown in figure 2B1 corre¬ 
sponds to the analytically proven model and serves as 
a validation of the numerical simulation. The cases 
in figure 2B2-4 are considerably more complex. In 
all these cases, the MI and the output AC peak at 
almost identical noise intensities, demonstrating the 
equivalence of the two measures. Even in the biolog¬ 
ically more plausible case of a temporally integrat¬ 
ing detector (i.e. the leaky integrate-and-fire neu¬ 
ron mm both, the output AC as well as the MI 
lead to similar optimal noise intensities (figure 2B5). 
In figure 3 we summarize a large number of inves¬ 
tigated models comprising different signal types, dif¬ 
ferent detector types and different signal-to-threshold 
distances. Here, the optimum noise intensities ac¬ 
cording to MI are plotted versus the corresponding 
intensities found by maximizing the output AC. Re¬ 
markably both measures are highly correlated (cor¬ 
relation coefficient r = 0.97). 

We conclude that the AC of the detector response 
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may serve as a universal, input independent objec¬ 
tive function to estimate the optimal level of noise 
in SR-like systems. This finding opens up a multi¬ 
tude of possible new technical implementations as it 
takes adaptive SR from theory to application. In ad¬ 
dition, adaptive SR based on output AC may explain 
a number of natural adaptive processes, especially in 
neural systems where this mechanism would be par¬ 
ticularly plausible as the computation of the AC may 
easily be implemented in neural networks |10j . Fi¬ 
nally, malfunctioning adaptive SR in neural sensory 
systems may be responsible for pathologic conditions 
like neuropathic pain or tinnitus, where the unsuc¬ 
cessful attempt to compensate for peripheral receptor 
damage leads to maladaptive central neural plastic¬ 
ity. 


Methods 

Mutual Information The mutual information 
I{S;Y), a measure frequently used in probability theory 
and information theory, quantihes the mutual depen¬ 
dence of two random variables S and Y. It determines 
how similar the joint distribution p{s, y) is compared 
to the product of the factored marginal distributions 
p{s)p{y) and is derived from Shannon’s entropy |15| . 

s y 

where p{s, y) is the joint probability distribution function 
of S and Y, and p(s) and p(y) are the marginal proba¬ 
bility distribution functions of S and Y respectively. For 
continuous random variables, the summation is replaced 
by a double integral: 

where p{s, y) is now the joint probability density function 
of S and Y, and p{s) and p{y) are the marginal proba¬ 
bility density functions. The natural unit of I{S;Y) is 
bits, however in some cases it might be more convenient 
to divide the total mutual information by the time or by 
the number of spikes within the observed spike train and 
thus derive mutual information rates R{S',Y) measured 
in bits s~^ or bits spike~^. The choice of the mutual in¬ 
formation as an objective function is natural, because the 


fundamental purpose of any sensor is to transmit infor¬ 
mation into a subsequent information processing system. 
Indeed, it has been shown by several authors [H1II1II3] 
that, within the context of stochastic resonance, I{S;Y) 
as a function of the variance cr^ of the added noise has a 
maximum that indicates the optimal level of noise. 

Signal-to-noise ratio The signal-to-noise ratio 
(SNR) in dB is given by the standard formula [TSlfTTl ISlI^ 

SNRds = 10 logjo {S/N{f)) (6) 

where N{f) is the amplitude of the noise power density 
at the stimulus frequency when presented alone, and S 
(signal power) is the area under the signal peak above the 
noise in the joint presentation case [9]. Remarkably, the 
above mentioned mutual information rate R is directly 
related to the non-logarithmic SNR through the formula 

poo Q 

'‘■I 

for gaussian distributions of signal and noise |15 |. 

Cross-correlation The normalized cross-correlation 
for time lag r = 0 of stimulus S and detector response Y 
is defined as 

^ _ {(st-s) {yt-y))t , . 

V{ist-sy)A{y.-yr),' ^ ^ 

where s and y are the means and {■}^ indicates averaging 
over time. 


Output autocorrelation The standard autocorre¬ 
lation as a function of the time lag r is defined as 


Cyy (■'") 


{{yt+r - y) {yt - y))t 

{(yt-yY)t 


(9) 


where y is the mean and (-jj indicates averaging over time. 
We note that for most applications (and discrete time 
steps) it is sufficient to consider only one time step, i.e. 
T = 1. However, for more complex signals, e.g. streams 
of n-bit words, it might be beneficial to calculate Cyy{T) 
for a number of different subsequent lag times. In order 
to derive a single value from the function, the root mean 
square (rms) of the autocorrelation function 


RMS(C,,) 



Yy^CyyirW 


( 10 ) 


may be calculated, where N-r is the total number of dif¬ 
ferent time lages. 
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Success probability The output of a memory-less 
sensor can be described by a conditional probability dis¬ 
tribution p{iit\st,nt), which includes deterministic be¬ 
haviour as a special case. Assuming statistically inde¬ 
pendent noise with distribution p{nt), the signal trans¬ 
mission properties of the sensor are given by p{yt\st) = 
'^„^piyt\st,nt)p{nt). Ideally, the sensor output should 
be equal to the input signal, yt = St, so that p{yt\st) = 
5yt,sf It is therefore meaningful to quantify the perfor¬ 
mance of a sensor by the success probability 

Q =p{yt = st), (11) 

which is expected to peak at some optimum noise level 
within the context of stochastic resonance. 

Analytical model In general, the momentary re¬ 
sponse of a SR-sensor can depend on the history of in¬ 
ternal states of the system, as is the case in integrate- 
and-fire-neurons. For simplicity, in the analytical model 
we only consider memory-less sensors, which respond to 
the present input signal st and noise value nt indepen¬ 
dently from their former activity states. 

We consider a bipolar stochastic sensor in which both 
the input signal St and the sensor output yt can only 
take on the values —1 and -|-1. The noise values nt, 
however, are continuous gaussian random numbers with 
variance and mean p — 0. We further assume that 
these two values appear in the input signal with equal 
probalility, p{st = —1) = p{st = -fl) = 0.5. By as¬ 
suming two symmetric detection thresholds ±6 (figure 3 
lower right inset), together with symmetric white noise, 
it can be assured that the distribution of sensor outputs 
p{yt = —1) = pijjt = +1) = 0.5 is also symmetric, so that 
the mean, variance and entropy of yt remain constant 
even if the noise level is changed. Hence, the expressions 
for /(S; Y) and Cyyij) can be slightly simplified. In par¬ 
ticular, the autocorrelation can be reduced to the non- 
normalized form Cyyij) oc (ytyt+r), and, furthermore, 
will be considered only for lagtime r = l. 

The sensor adds the noise nt to the binary input signal 
St- If st+nt exceeds the upper threshold 6, the output yt 
is -fl, if St+nt falls below the lower threshold —9, output 
yt is —1. For St + nt € [—9,+9], the output is chosen 
randomly between the two binary values -1-1 and —1. 

We are interested in the case of a threshold 9 > 1 
which exceeds the signal amplitude, so that without the 
assistance of added noise the signal cannot be detected. 
Adding a random noise value nt to a (say) positive input 
signal St can have three possible effects. If we consider 


the noise to be sufficiently positive to lift the signal be¬ 
yond the upper threshold, then the success probability 
Q = P{yt = -1-1 I St = -1-1) = piyt = -1 I St = -1) will 
be increased. Alternatively, if the noise happens to be 
strongly negative and draws the positive signal below the 
lower threshold —9 then the success probability Q will be 
decreased. The third possibility is that St + nt remains 
sub-threshold. Such cases make the signal transmission 
neither better nor worse. 

It is intuitively clear that small noise levels will increase 
Q, but as soon as a considerable fraction of momentary 
noise levels nt exceeds 2+ (6—1), the success probability 
Q will fall again. In our case it is given by Q = Q((t) = 
i + U Wi^) - W(^) ] , where W(®) = |erf(^) is a 
slightly rescaled error function (see Derivation of success 
probability for a detailed derivation). As a function of the 
noise level a, the success probability has a well-defined 
maximum. 

In this sensor model, the mutual information !(¥; S) 
can be expressed as a strictly increasing function of the 
success probability: /(Q) = 1-f Q logj Q-I-(1—Q) log 2 (l— 
Q) (see Derivation of mutual information for a detailed 
derivation). 

Since both / and Q require access to the sub-threshold 
signal St, we turn to the autocorrelation function Cyy of 
the sensor output. Since the mean y of yt is zero and 
its variance constant, we can use a non-normalized ver¬ 
sion of equation(6). Furthermore, we restrict our ana¬ 
lytical consideration to a single lag-time t = 1, defining 
C = {ytyt+i}- The modulus of this quantity, too, can be 
expressed as a strictly increasing function of the success 
probability: |C'(Q)| = | (stSt+i) | [1 — 4Q(1 —Q)], where 
(stSt+i) are the input correlations (see Derivation of out¬ 
put autocorrelation for a detailed derivation). 

Derivation of success probability The normal¬ 
ized Gaussian distribution with zero-mean and standard 
deviation cr is given by 

9ix,(j) = ( 12 ) 

V zTrcT 

For later convenience, we define a function W (a:) via 

where erf(a;) = fo^~* d-t is the error function. 

The success probability Q is given by 


4 



Q 


2I{Y;S) 


= p{yt =+l\st =+i) = 

= ^■pi—9 — i<nt<6—l) + 

+ p(nt > 0 — 1) (14) 


The factor | accounts for the stochastic output of the 
unit in the case when st + nt is sub-threshold. We can 
now express the probabilities as integrals over Gaussians: 


Q = 


-b 


f0+i 


g{x,a)dx+ / g(x,a)dx] + 


g(x,a)dx] (15) 


Next we use the function W{x) defined above: 


Q = 


-b 



(16) 


Derivation of mutual information The mutual 
information of the detector output and the input signal 
is defined as 


I{Y-S) 


^ p{v, s) log 2 

y,s 


f p{y,s) \ 
\p{y)p{s)J 


P(yl«)p(«) log2 

y,s 


/ p(yls)p(s) \ 

V p{y)p{s) ) 


^ p(2/|s)(V 2) loga 

y,s 


f p{y\s){l/2) \ 

\ (1/2)(1/2) J 


I iog2(2p(j/|s)). 

y,s 


(17) 


= (2j3(y|s)) = 

y,s 

= p(y = -l|s = -l) log 2 (2p(i/ = -l|s = -l))-b 
+ J5(t/ = -l|s = +l) log 2 (2p(l/=-l|s = +l)) -b 

+ p(y = + l|s = -l) log 2 (2p(i/ = -bl|s = -l))-b 
+ J5(t/ = +l|s = +l) log 2 (2p(l/=-bl|s = -bl)) = 

= Q loga (2Q) -b 

+ (1 “ Q) log2 (2(1 — Q)) + 

+ (1 “ Q) loga (2(1 — Q)) + 

+ Q log 2 (2g). (18) 


Therefore 


/(T; S) = Q loga (2Q) + (1 - Q) loga (2(1 - Q j) 

= 1 + Q log 2 (Q) + (1 - Q) log 2 (l - QlJ-9) 


Derivation of output autocorrelations in the 
analytical model The temporal correlations of the 
input signal can be expressed by the probability q = 
p(si=-bl,so = -bl) in the following way: 


(st+l St) = (si So) = 

= (®1 ®o) = 

SO)-®! 

= p(si = -l|so = -l)p(so = -l) [(-1)(-1)] + 

-b p(si=-l|so = -bl)p(so = -bl) [(-!)(+!)] + 

+ p(si = -bl|so = -l)p(so = -l) [(+!)(-!)] + 

+ p(si=-bl|so = -bl)p(so = -bl) [(+!)(+!)] = 

= 9(1/2)[11 + 

+ (l-g)(l/2)[-l] + 

+ (l-g)(l/2)[-l] + 

+ q (1/2) [1] = 2q-l. (20) 

The temporal correlations in the output signal are given 


Cyy{T =1) = {yt+1 yt) = {yi yo) = 

= piyi’Vo) (yiyo)- ( 21 ) 

MO.HI 


We explicitly go through all four terms: 
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Consider for example the probability p{yi = +l,yo = 
-bl). There are four different chains of events which can 



produce a sequence of two successive +l’s in the output 
signal: 

p{yi=+l,yo = +l) = 

= p(l/l =+l|si =-l)p(si =-l|so = -l) ■ 

• p(i/o = +l|so = -l)p(so = -l) + 

+ p(l/l =+l|si =-l)p(si =-l|so = +l) • 

• p(i/o = +l|so =+l)p(so =+l) + 

+ p(j/l =+l|si =+l)p(si =+l|so = -l) ■ 

• f’(j/o = +l|so = -l)p(so = -l) + 

+ p(yi = +l|si =+l)p(si =+l|so = +l) ■ 

• p(yo = +l|so =+l)p(so =+l) = 

= (1-Q)?(1-Q) (1/2) + 

+ (1 - Q) (1 - g) Q (1/2) + 

+ Q(l-g) ( 1 -Q) ( 1 / 2 ) + 

+ QqQ (1/2) = 

= | + (2g-l)g(l-Q)-A (22) 

For symmetry reasons, p{yi = —l,yo = —1) = p{yi = 
+l,yo = +1) = A. In the same way, p{yi = +l,t/o = 
-1) = P{yi = -1,1/0 = +1) = B. 

Since y^Piyi,yo) = 1 = 2A + 2B, it follows that 
B^\-A. 

Knowing all four joint probabilities, we can proceed to 
compute the temporal correlations in the output signal: 


Cyy{r = 1 ) 


= (i/i yo) = 

= +(-l)(-l)+B(-l)(+l) + 

+ i3(+l)(-l) + yl(+l)(+l) = 

= 2A-2B = 


= {2q-l)[l-iQ{l-Q)] = 

= {st+i St) [ 1 - 4Q(1 - g) ]. (23) 


Input signals for numerical simulations Both, 
synthetically generated as well as natural signals were 
used in numerical simulations. As a discrete input signal, 
a correlated, bipolar string st € { —1,+1} was generated, 
in which the probability of successive values being iden¬ 
tical was Prob(st = St_i) = 0.7. As continuous signals 
in order of increasing complexity we used: first, a sine 


waveform signal with constant frequency and amplitude; 
second, an aperiodic time series derived from the variable 
x(t) of the Roessler attractor [16] 

» = -(y + z) 

y = x + ay 
z = h + [x — c)z, 

(24) 

with parameters a = 0.15, b = 0.2 and c = 7.1; third, 
the aperiodic random Ornstein-Uhlenbeck process [17] 
X = —^x + e ^{t), where ^ is an independent normally dis¬ 
tributed random variable, r is the correlation time and e 
is the noise amplitude; fourth, wave files of speech, music 
and natural sounds have been recorded. All synthetically 
generated signals were computed by numerically integrat¬ 
ing the differential equations using fourth order Runge 
Kutta method. 

Sensor models for numerical simulations Four 
different memory-less sensor models were implemented by 
combining symmetric and asymmetric thresholds, with 
discrete and continuous sensor output functions. In the 
symmetric models, there exist two thresholds +9 and 
—9. Without added noise, the sensor output is zero for 
I St I < 9, St—9 for St > 9, and st + 9 for st < —9. In 
the asymmetric models, there exists only a single positive 
threshold 9 > 0. Here, without added noise, the sensor 
output is zero for St < 9 and St—9 for st >= 9. We note 
that the analytical model described above belongs to the 
class of discrete symmetric models. As one example of 
the detectors with memory, were the output depends not 
only on the momentary input and added noise, but also 
on earlier internal states of the detector itself, we choose 
the leaky integrate-and-hre neuron model mm with 

1 

X = -a: + St 

'T~m 

(25) 

where x is the membrane potential, Tm the mebrane time 
constant and St the input signal. If x crosses the threshold 
9 from below, an output spike is generated and the mem¬ 
brane potential is set to the resting potential Xr, which is 
chosen to be zero for simplicity. 
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signal-to-noise ratio SNR(s,y) 
mutual information l(s,y) 

_.oss C‘ 

output autocorrelation Cyy 


Figure 1: Sketch of the adaptive stochastic resonance principle. In case, where the underlying signal is known, all 
objective functions can be calculated, whereas if the signal to be detected is unknown the only objective function that 
can be computed is the output autocorrelation. 
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Figure 2: A.- Sample signals used for numerical simulations. As a discrete signal a correlated bipolar chain (Al) 
was chosen. As continouos aperiodic signals served, in order of complexity, one dimension of the 3-dimensional 
Roessler attractor (A2), an Ornstein-Uhlenbeck process (AS) and a wave file of recorded speech (A4). A simple 
sinosoid signal (A5) was generated as continuous periodic input. B: Sample results of numerical simulations, i.e. 
resonance curves of mutual information (red lines) and output autocorrelation (blue lines) for different models. The 
numbering corresponds to that of subfigure (A) and indicates the underlying input signals. The letters refer to the 
detector models with continuous/discrete output (C/D) and asymmetric/symmetric threshold (A/S). The case shown 
in (Bl) corresponds to the analytical model. In addition to the numerical results, the analytical predictions of mutual 
information (orange line) and output autocorrelations (cyan line) are plotted. (B5) is a sample result of the integrate- 
and-fire neuron model. 




















optimum noise intensity according to Ml 


Figure 3: Scatter plot that summarizes a large number of investigated models comprising different signal types, 
different detector types and different signal-to-threshold distances. The optimal noise intensities according to mutual 
information (MI) are plotted versus the corresponding intensities found by maximizing the output autocorrelation 
(AC). Remarkably both measures are highly correlated (correlation coefficient r = 0.97/ 


9 






























References 

[1] Roberto Benzi, Alfonso Sutera, and Angelo Vulpiani. 
The mechanism of stochastic resonance. Journal of 
Physics A: mathematical and general, 14(11):L453, 
1981. 

[2] Anthony N Burkitt. A review of the integrate-and- 
hre neuron model: I. homogeneous synaptic input. 
Biological cybernetics, 95(1):1"19, 2006. 

[3] JAMES J Collins, THOMAS T Imhoff, and PETER 
Grigg. Noise-enhanced information transmission in 
rat sal cutaneous mechanoreceptors via aperiodic 
stochastic resonance. Journal of Neurophysiology, 
76(l):642-645, 1996. 

[4] JJ Collins, Carson C Chow, Thomas T Imhoff, 
et al. Stochastic resonance without tuning. Nature, 
376(6537):236-238, 1995. 

[5] John K Douglass, Lon Wilkens, Eleni Pantazelou, 
Frank Moss, et al. Noise enhancement of information 
transfer in crayfish mechanoreceptors by stochastic 
resonance. Nature, 365(6444):337-340, 1993. 

[6] A Aldo Faisal, Luc PJ Selen, and Daniel M Wolpert. 
Noise in the nervous system. Nature Reviews Neu¬ 
roscience, 9(4):292-303, 2008. 

[7] Luca Gammaitoni, Peter Hanggi, Peter Jung, and 
Fabio Marchesoni. Stochastic resonance. Reviews of 
modern physics, 70(1):223, 1998. 

[8] L Lapique. Recherches quantitatives sur I’excitation 
electrique des nerfs traitee comme une polarization. 
J Physiol Pathol Gen, 9:620-635, 1907. 

[9] Jacob E Levin and John P Miller. Broadband 
neural encoding in the cricket cereal sensory sys¬ 
tem enhanced by stochastic resonance. Nature, 
380(6570):165-168, 1996. 

[10] JCR Licklider. A duplex theory of pitch perception. 
The Journal of the Acoustical Society of America, 
23(1):147-147, 1951. 

[11] Hiroyuki Mino. The effects of spontaneous random 
activity on information transmission in an auditory 
brain stem neuron model. Entropy, 16(12):6654- 
6666, 2014. 

[12] Sanya Mitaim and Bart Kosko. Adaptive stochastic 
resonance. Proceedings of the IEEE, 86(11):2152- 
2183, 1998. 

[13] Sanya Mitaim and Bart Kosko. Adaptive stochastic 
resonance in noisy neurons based on mutual infor¬ 
mation. Neural Networks, IEEE Transactions on, 
15(6):1526-1540, 2004. 


[14] Frank Moss, Lawrence M Ward, and Walter G San- 
nita. Stochastic resonance and sensory information 
processing: a tutorial and review of application. 
Clinical Neurophysiology, 115(2):267-281, 2004. 

[15] Claude Elwood Shannon and Warren Weaver. The 
mathematical theory of communication. University 
of Illinois Press, 1959. 

[16] G Wenning and K Obermayer. Activity driven adap¬ 
tive stochastic resonance. Physical review letters, 
90(12):120602, 2003. 

[17] Kurt Wiesenfeld, Frank Moss, et al. Stochastic res¬ 
onance and the benefits of noise: from ice ages to 
crayfish and squids. Nature, 373(6509):33-36, 1995. 

[18] Ting Zhou, Frank Moss, and Peter Jung. Escape¬ 
time distributions of a periodically modulated 
bistable system with noise. Physical Review A, 
42(6):3161, 1990. 


10 



